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children, primarily those classified as educable mentally retarded 
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Introduction to Evaluation of Activities Ending June 15 j 1971 



For a period of two years, Septeinber , 1969 to June, 1971 j the Media Utili- 
zation Services for Teachers (MUST) project has heen working toward the objective 
of developing and demonstrating "a systems approach to facilitate the utiliza- 
tion of media to improve instruction for children, primarily those classified 
as EMR and functionally retarded" (3, p.ll+).* The project personnel characterized 
the task as one of developing a "systems, approach to enable the schools to 
analyze learner needs and redesign curricula in terms of their needs. (3, 
p.ll+). They further stated, "The development of clearly defined instruction 
objectives, the analysis of learning tasks, the matching of appropriate media 
to instructional objectives will result in the development of replicable vali- 
dated instiuictional procedures and materials" (3, p.lU). 

At the beginning of the school year 1970-71 » the Must project staff had 
produced, after an eleven-month period of planning and initial implementation, 
a plan to establish media centers and specialized instruction for educable 
mentally retarded (EMR) and/or functionally retarded children in three schools, 
with major implementation centering at the C. W. Hill School. Equipment and 
limited consultant services were provided for the other two t£n*get schools, 

J. M. Jones and L. P. Miles. Because only equipment and occasional consultant 
services were provided to the latter two schools and no documentation had been 
made by the project staff, no attempt was made to assess the MUST activities 
in the schools. 

It should -be noted that this report is presented at a point in time that 
* See Bibliography. 
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marks the end of the major developmental work and pilot implementation of what 
might be termed a "prototype program. " The summer of 1971 has focused on 
refinement of the model and preparation for full implementation of a modified 
program that will undergo continuing development and modification during the 
1971-72 school year. 

The integral and tangential factors which have influenced the progress 
of the Media Utilization Services for Teachers (MUST) during the 1970-71 school 
year have been the major focus of the investigation. The assessment aspect 
has included a review of the history of the project as documented in MUST pro- 
posals and reports, interview-discussions with the personnel having the major 
responsibilities fox' directing, designing, and implementing the project, and 
analysis of the data collected by the project’s personnel. This report in- 
cludes a review of the implementation procedures, identification of problems 
encountered, and presentation of a limited set of data, available for analysis. 
Evaluative comments are, in the main, of the formative type; that is, the 
evaluative statements are assessments about the progress attained by the pre- 
ceding operational procedxires. Because of the developmental nature of the 
project, suinmative eval\iation is limited; but such evaluation will encoo5)ass 
comments concerning the extent to which particular objectives have been attained 
and the collection of data that provides objective support for the impact of the 
program. 



MUST Objectives, Operations and Evaluative Comments 

Objectives . The specific objectives the MUST project has set for itself 
have, in the course of two years, been modified. In the initial proposal for 
the 1971-72 school year the objectives were stated as follows (3> p. 7): 
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1. Evaluate traditional methods for diagnosing children as EMR. 

2. Establish a system and methods for diagnosing poor performance 
in academic areas. 

3. Utilize diagnostic results and identify the appropriate media 
to arrive at a comprehensive prescription for each child with 
the ultimate goal being individualized instruction. 

4. Design and modify media to aid in individualized instruction. 

5 . Extend the skills of resource and classroom teachers, and train 
teacher aides that comprise the MUST team in the three model 
schools in the utilization of media identified or developed. 

6. Significantly improve performance in those academic areas for which 
individualized prescriptions have been written. 

7 . Analyze cost effectiveness of the MUST project and prepare a 
plan for extending MUST objectives into additional schools in 
the Atlanta School System. 

8. Develop a model for national dissemination which will provide 
specific guiddines for implementing MUST objectives into any 
school. 

These objectives, written in January, 1971> represented the objectivesof 
the activities of the MUST project during the 1970-71 school year. The attain-' 
ment of the objectives were, however, only partially met and, as shall be 
documented later, were modified in a revised proposal for the third year’s 
operation. 

Objectives I- 3 . Consideration of the first three specific objectives 
revealed the following observations. First, the objectives were generated 
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by the realization, during the project’s first planning year, that one major 
problem v;as that of identifying students whose ability range fell into that 
labeled "educable mentally retarded'.' Because a high percentage of children 
in the target schools were low achievers on the Metropolitan Achievement Test, 
some means of differentiating those children who definitely fell in the EMR 
ability range from those who were not, but had not acquired the achievement 
levels commensurate with their potential, needed to be identified. The latter 
group has been referred to as "functionally retarded" in the MUST documents. 

The differentiation problem was compounded by the facts that (l) the special 
psychological services required to determine whether or not a child qualified 
for a program restricted to the educable mentally retarded were not available 
for all the possible candidates at C. W. Hill; (2) there was some doubt con- 
cerning the validity of the testing services in making the required differen- 
tiations, particularly among children from low socio--economic hones; and (3) 
selection of students had to be made during the first month of the school year. 

The project staff, in cooperation with the Exceptional Children Services 
staff, proceeded to establish a means to "find new ways of dealing with the 
proper identification and instruction of slow learners in large metropolitan 
school systems" (2, p. 17). The development of a procedure to identify the 
children to be included in the first year’s target group relied on the use 
of selected test scores and judgments by the teachers and principal of C. W. 
Hill School and the project staff. The procedure included: 

(1) assignment of those children for whom intelligence test 
scores were available and fell within the 50-75 intelligence 
score range to two self-contained classrooms with special 
education teachers; 

(2) identification of 110 students in grades two through five 
as the slowest learners in each grade; 



(3) assignment of the identified slow learners to one of the 
four regular classrooms for which a MUST team teacher had 
the teaching responsibility; 

(U) selection of the pilot sample: 20 to 32 children whose 

test scores on the screening tests selected by the MUST 
staff and personnel ^judgments indicated that the sample 
was composed of the five to eight slowest learners in each 
classroom. 

The initial diagnostic procedure was one of testing approximately l40 
children enrolled in grades two throijgh five and classified as slow learners 
by the staff at C. W. Hill School. Early in the 1970-71 school year two 
diagnostic tests, the Hoijghton-Mifflin Pre-reading Inventory, Part II and 
the Basic Skills Checklist, constructed by the MUST staff were administered 
to the selected sait 5 )le. The two tests made possible the evaluation of students 
on basic pre- and initial reading tasks. The initial data indicated great 
variability within each student’s performance on the subtests and among the 
students at each grade level. The first 1970-71 quarterly report (2) reports 
the data by subject, grade and EMR group. The mean scores indicated an in- 
crease in ability on each of the subtests across grade level, with the EMR 
class obtaining scores slightly below the fourth and fifth grade groups. The 
distribution of scores also indicated that on many of the subtests a large 
percentage of children were obtaining near or almost perfect scores on each 
subject. The distributions tended to become constricted across the grade levels, 
suggesting that as basic diagnostic tool, the tests were capable of obtaining 
a distribution of scores that reflected the lowest levels at which the students 
were working at the time of testing. However, the accumulation of scores at 
or near perfect levels also suggested that the basic diagnostic batteries in- 
tended to serve as pre- and post-test measures for at least part of the target 
population, had a low ceiling and therefore, restricted the assessment of 
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adequate growth data for all those pupils who obtained relatively high scores 
on some of the subtests. 

The children in the initial testing pool were separated into five 
groups: (1) assignment to one of two self-contained EMR classrooms (EMR I, 

EIv® II); (2) selection for daily work of approximately one and one-half hours 
in the resource room (Resource Room); ( 3 ) selection for work in a Basic Skills 
laboratory (Basic Skills); and (^) no special attention concept for being 
assigned to one of the four teachers whose responsibilities included that of 
teaching a "regular class" and being a member of the MUST team (No Special 
Attention). The target pop\ilation for the MUST project were those children 
selected for work in the Resource Room. One teacher, a special education 
teacher, maintained the responsibilities of implementing, testing, and assis- 
ting in the development of the procedures for teaching the selected students. 
Major emphasis was given to the development of reading skills. The children 
selected for work in the Basic Skills Laboratory were a second "target" sample; 
On the basis of the test scores, the children were directed by a teaching aide 
to the media that had been identified as means to acquire a specific objective. 
The set of objectives used were derived from the skills required to operate 
specific pieces of equipment and the abilities assessed by the pre-tests. 

The remaining groins had no direct contact with the MUST program. Some may 
have been exposed to certain procedures; however teacher-interest in the 
materials and methods available through the MUST work varied from no to moderate 
involvement and no assessment of impact on the three groups was possible. 

Pre-test data by assignment and grade level are presented in Tables 1 and 
2. Table 1 contains the mean score and range, of scores for each group and grade 
level on the subtests of the Houghton-Mifflin Pre-reading Inventory, Part II. 



Tatle 2 contains similar data for the suhtests of the Basic Skills Test and 



when available, intelligence test scores. 
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Fre-^st Means and Range of Scores on the Basic Skills ^st 
and Wechslar Intelligence Test for Children by Group and Grade Level. 





'^Bata not available 
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TABLE 2 i 



Pre-test Means and Range of Scores on Houghton-Mifflin Pre-reading Test by 

Treatment Groups and Grade Level 



Group and Grade Level 


Subtests 
















Matching 








Using 


Finding 


Listening 


Letters anc 


Total 




Context 


Letters 


for Letters 


Sounds 


Score 




X 


Range 


. X 


Range 


X 


Range 


. X 


Range 


X 


Range 


I. Resource Room 






















Grade 2 (N= 6 ) 


6.5 


4-8 


12.5 


8-16 


7.3 


4-11 


8.7 


7-11 


35.0 


30-43 


Grade 3 (N=6) 


7.0 


4-8 


13.7 


9-18 


6.5 


5-14 


7.8 


4-17 


35.0 


27-56 


Grade 4 (N=5) 


6.4 


5-8 


15.6 


l4-l8 


8.8 


6-14 


10.8 


8-16 


41.6 


36-55 


II. Basic Skills 






















Grade 2 (N=H) 


7.0 


3-8 


14.3 


13-18 


6.2 


6-13 


8.9 


4-17 


36.4 


26-54 


Grade 3 (N=9) 


6.6 


3-8 


14.2 


9-16 


6.1 


6-10 


7.1 


6-9 


34.0 


27-40 


Grade 4 (N=2) 


7.0 


6,8 


18.0 


18 


8.5 


8,9 


14.5 


14,15 


48.0 


47,49 


III. No Special Treatment 






















Grade 2 (N=8) 


6.9 


5-8 


14.3 


6-17 


5.5 


4-9 


9.5 


8-11 


36.2 


25-40 


Grade 3 (N=9) 


7.0 


6-8 


17.0 


16 - 1 B 


11.0 


5-14 


11.9 


5-17 


48.5 


34-57 


Grade 4 (N=12) 


6.9 


6-8 


17.7 


17-18 


11.8 


6-14 


16.3 


10-lS 


56.3 


52-58 


Grade 5 (N=1?) 


7.8 


6-8 


17.8 


17-18 


13.5 


12-14 


17.1 


16-18 


48.3 


28-57 


IV. EMR I (N=16) 


7.4 


4-8 


16.6 


l4-l8 


10.4 


4-14 


13.9 


8-18 


48.3 


28-57 


V. EMR II (N=3) 


7.0 


6-8 


17.0 


16-18 


13.0 


12-14 


15.3 


14-17 


52.3 


49-57 
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When the children's scores are separated into subgroups, the tendencies 
observed in the quarterly report data do not seem to hold up as well. While 
the range restriction remains, there is little variation between grade level 
and assignment groups. The sample for whom the data was available totals 
114 students, leaving approximately 36 sets of scores unaccounted for. In 
addition, only 1? sets of scores are available of the twenty to thirty- two 
students reported to have been selected for. instruction in the Resource Roan. 
Judging from the pre-test data, very little differentiation is apparent. The 
children receiving no special attention appear to have achieved slightly higher 
scores, thereby, confirming the differential selection process. The differences 
in means and range are not large, indicating that the selected tests did not 
tap higher ranges of reading ability. 

The MUST staff considered further assessment, as well as sd.ection of some 
control or comparison groups. The priorities, as viewed by the majority of 
the staff, did not include intensive assessment or monitoring of the progress 
of selected or control children. Individual progress of pupils attending either 
the Resource Room or the Basic Skills Laboratory received major attention, as 
the second 1970-71 quarterly report indicates. Unfortunately, overall co- 
ordination-documentation of scores on the pre-test and instruction as guided 
by the behavioral objectives were not made, nor was it possible to reconstruct , 
the matching procedure of learner abilities and instructional objectives. 

Progress, as indicated by testing procedures was again limited by several 
factors. Only those children assigned to the Basic Skills Laboratory or the 
Resource Room were post-tested. Testing in all three schools in April and May, 
1971 , had the primary objective of obtaining data to serve as selection infor- 
mation. Once the target populations for each of the three schools had been 
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identified, the data also served as baseline data for planning specific in- 
structional objectives for each child to attain during the 1971-72 school year. 
Because the project staff decided to place major emphasis on younger children 
who still faced the problems of accomplishing the initial stages of learning 
to read, only two children in the groip who received no special attention 
during the 1970-71 were in the spring testing group. None of the EMR classes 
were post-tested. Thus, no comparisons between groups could be made. 

The assessment of change in the Resource Room and Basic Skills groups 
was made by examining the change in scores from pre- to post-testing. Because 
of the skewed distributions, arbitrary cut-off points were established and 
the frequencies of subjects falling within the inv-ervals determined by the 
cut-off points at pre- and post-testing were examined. The cut-off points 
v;ere set to identify the lower 25 ^, middle 50 ^ &nd upper 25 ^^ of the total 
possible points. The distributions of scores by grade (2, 3> and by group 
(Resource Room and Busic Skills) were examined, Fisher's Exact Probability 
Test (Seigel, 1956, p. 96 - 8 ) was used to determine whether not there was a 
significant change in the distributions from pre- to post-test. Tables 3-9 
present the distribution of pre- and post-test scores on each of the subtests 
of the Basic Skills Checklist, Total score distributions are included in 
Table 10. Similar data for the subtests and total scores for the Houghton- 
Kifflin Pre-reading Inventory, Part II are found in Tables U-I 5 . 



TABLE 3 

Color Name, Nonverbal, Basic Skills Checklist: 
Pre- and Post-test Distributions by Grade and Group 



Grade 


Group 


N 


Test Time 


Score 


1 Interval 


P* 










0-2 


3-9 


10-12 




2 


Basic Skills 


10 


Pre 


0 


1 


9 










Post 


0 


0 


10 


0 

lf\ 

• 




Resource Room 


7 


Pre 


0 


3 


k 










Post 


0 


0 


7 


.10 




Basic Skills 


9 


Pre 


0 


1 


8 










Post 


0 


0 


9 


.50 




Resource Rocan 


5 


Pre 


0 


1 


k 










Post 


0 


0 


5 


.50 




Basic Skills 


2 


Pre 


0 


0 


2 










Post 


0 


0 


2 


1.00 




Resource Room 


5 


Pre 


0 


8 


30 










Post 


0 


0 


38 


N.C.** 




Total Group 


38 


Pre 


0 


8 


30 










Post 


0 


0 


38 


N.C. 



* p = probability of distributions differing between pre- and post-test 
using Fisher's Exact Probability Test. 

** N. C. = Not computed 
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TABLE k 



Color Naifle, Verbal, Basic Skills Checklist; 
Pre- and Post-test Distributions by Grade and Group. 



Grade 


Group 


N 


Test time 


Score Interval 
0-2 3-9 10-12 


P 


2 


Basic Skills 


10 


Pre 


0 


2 


8 










Post 


0 


0 


10 


N.C. 




Resource Roan 


7 


Pre 


0 


5 


2 










Post 


0 


2 


5 


.Ik 



3 


Basic Skills 


9 


Pre 


0 


k 


5 










Post 


0 


1 


8 


.15 




Resource Room 


5 


Pre 


0 


k 


1 










Post 


0 


0 


5 


.02 



k 


Basic Skills 


2 


Pre 


0 


0 


2 










Post 


0 


0 


2 


N.C. 




ResoTirce Room 


5 


Pre 


0 


2 


3 










Post 


0 


0 


5 


CVI 

CVJ 

• 




Total Group 


38 


Pre 


0 


15 


21 










Post 


0 


7 


35 


N.C. 
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TABLE 5 



Shape Name, Nonverbal, Basic Skills Checklist: 
Pre- and Post-test Distributions by Grade and Group. 



Grade 


Group 


N 


Test Time 


Score Interval 
0-2 3-6 10-12 


P 


C 


Basic Skills 


10 


Pre 


0 


5 5 










Post 


0 


0 10 


.02 




Resource Room 


7 


Pre 


0 


5 2 










Post 


0 


1 6 


.05 



3 Basic Skills 


9 


Pre 


0 


5 


k 








Post 


0 


0 


9 


.02 


Resource Room 


5 


Pre 


0 


3 


2 








Post 


0 


0 


5 


.08 



h Basic Skills 


2 


Pre 


0 


0 


2 


i 

j 






Post 


0 


0 


2 


N.C. ; 


Resource Room 


5 


Pre 


0 


1 


4 . 


} 

i 






Post 


0 


0 


5 


.50 j 

1 


1 

Total Groi^ 


38 


Pre 


0 


19 


19 








Post 


0 


1 


37 


N.C. 
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TABLE 6 



Shape Name, Verbal, Basic Skills Checklist: 

Pre- and Post-test Distributions by Grade and Group. 



1 


Grade 


Group 


N 


Test Time 


Score Interval 


P 


i 










0-2 3-6 7-8 





Basic Skills 



Resource Room 



10 



Basic Skills 



Resource Room 



9 

5 



Pre 

Post 

Pre 

Post 



Pre 

Post 

Pre 

Post 



0 

0 

1 

0 



0 

0 

0 

0 



8 

2 

0 



5 

0 

If 

1 



2 

8 

3 

7 



If 

9 

1 

If 



.2lf 

.03 



.02 



.10 
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If 


Basic Skills 


2 


Pre 


0 


0 


2 










Post 


0 


0 


2 


N.C. 




Resource Room 


5 


Pre 


0 


If 


1 










Post 


0 


0 


5 


• 

0 

fO 




Total Group 


38 


Pre 


1 


2if 


13 










Post 


0 




35 


N.C. 




TABLE 7 



Number Identification, Basic Skills Checklist: 
Pre- and Post-test Distributions by Grade and Group. 



Grade 


Group 


N 


Test Time 


Score 

0-2 


Interval 
3-7 8-10 


P 


2 


Basic Skills 


10 


Pre 


0 


2 


8 










Post 


0 


0 


10 


CM 




Resource Room 


7 


Pre 


1 


0 


6 










Post 


1 


0 


6 


.77 





Basic Skills 


9 


Pre 


0 


2 


7 










Post 


0 


2 


7 


.71 




Resource Room 


5 


Pre 


1 


2 


2 










Post 


0 


1 


k 


VO 

CM 

• 


k 


Basic Skills 


2 


Pre 


0 


0 


2 










Post 


0 


0 


2 


N.C. 




Resource Room 


5 


Pre 


0 


0 


5 










Post 


0 


0 


5 , 


1.0 




% 

Total Group 


38 


Pre 


2 


6 


30 










Post 


1 


3 


3^ 


N.C. 
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Number Concept, Basic Skills Checklist: 

Pre- and Post-test Distributions by Grade and Group. 



t/ 



Grade 


Group 


N 


Test Time 


Score 


Interval 


P 










0-1 


2-5 


6-7 




2 


Basic Skills 


10 


Pre 


0 


2 


8 


■ N. 








Post 


0 


0 


10 


.2k 




Resource Room 


7 


Pre 


0 


1 


6 










Post 


0 


1 


6 


.77 




Basic Skills 


9 


Pre 


0 


2 


7 










Post 


0 


1 


8 


.50 




Resource Room 


5 


Pre 


0 


1 












Post 


0 


1 


k 


.78 


k 


Basic Skills 


2 


Pre 


0 


0 


2 










Post 


0 


0 


2 


N.C. 




Resource Room 


5 


Pre 


0 


0 


5 






« 




Post 


0 


0 


5 


1.0 




Total Group 


38 


Pre 


0 


6 


32 










Post 


0 




35 


N.C. 



6 

ERIC 



V y y *i 
^ » 



21 
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TABLE 9 

Letter Identification, Basic Skills Test; 

Pre- and Post-test Distributions by Grade and Group. 



Grade 


Group 


N , 


Test Time 


Score Interval 


P ■ 










0-13 11^-59 


40-52 




2 


Basic Skills 


10 


Pre 


3 


k 


3 










Post 


0 


1 


9 


.01 




Resource Room 


7 


Pre 


2 


k 


1 










Post 


0 


0 - 


7 


.002 


3 


Basic Skills 


9 


Pre 


2 


k 


3 










Post 


0 


2 


7 


.08 




Resource Room 


5 


Pre 




0 


2 










Post 


0 


1 


4 


.26 


4 


Basic Skills 


2 


Pre 


0 


0 


2 










Post 


0 


0 


2 


N.C. 




Resource Room 


5 


Pre 


1 


1 


3 










' Post 


0 


0 


5 


.22 




Total Group 


38 


Pre 


11 


13 


14 










Post 


0 


k 


34 


N.C. 



I 

I 




- 18 - 




4 






a 



TABLE 10 



Total Performance Score, Basic Skills Test: 

Pre- and Post-test Distributions by Grade and Group. 



Grade 


Group 


N 


Test Time 


Score Interval 

0-27 28-81 82-109 


P 


2 


Basic Skills 


10 


Pre 


0 


6 


4 










Post 


0 


0 


10 


.005 




Resource Room 


7 


Pre 


0 


5 


2 










Post 


0 


0 


7 


.01 



3 


Basic Skills 


9 


Pre 


0 


5 


k 










Post 


0 


1 


8 


.07 




Resource Room 


5 


Pre 


0 


3 


2 










Post 


0 


1 


k 


VO 

OJ 

• 



k 


Basic Skills 


2 


Pre 


0 


0 


2 










Post 


0 


0 


2 


N.C. 




Resource Room 


5 


Pre 


0 


1 


4 










Post 


0 


0 


5 


.50 



38 Pre 0 20 iB 

Post 0 2 36 N.C. 



Total Group 



TABLE 11 



Using Context, Bought on -Mifflin Prereading Inventory, Part II: 
Pre- and Post-test Distributions by Grade and Group. 



Grade 


Group 


N 


Test Time 


Score Interval 


P 










CVJ 

1 

o 


3-6 


7-8 




2 


Basic Skills 


10 


Pre 


0 


2 


8 










Post 


0 


1 


9 


.50 




Resource Room 




Pre 


0 


2 


4 










Post 


0 


0 


6 


00 

CVJ 

• 


3 


Basic Skills 


9 


Pre 


0 


3 


6 










Post 


0 


2 


7 


.50 




Resource Room 




Pre 


0 


1 


5 










Post 


0 


0 


6 


.50 


k 


Basic Skills 


2 


Pre 


0 


1 


1 










Post 


0 


0 


2 


.50 




Resource Room 


5 


Pre 


0 


2 


3 










Post 


0 


1 


4 


.50 




Total Group 


38 


Pre 


0 


11 


27 










Post 


0 


h 


34 


N.C. 



* N is 1 less than N used in Basic Skills Checklist analysis. 

** N is 1 more than N used in Basic Skills Checklist analysis. 
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TABLE 12 



Finding Letters, Hoi:ighton-Mifflin Pre-reading Inventory, Part II: 



> 

} 



Pre- and Post-test Distributions by Grade and Group. 



■3 



Grade 


Group 


N 


Test Time 


Score Interval 


p 5 










0~h 




15-18 


1 

:! 


2 


Basic Skills 


10 


Pre 


0 


5 


5 


1 

i 








Post 


0 


0 


10 


.02 j 




Resource Room 


6 


Pre 


0 


3 


3 


» 

i 








Post 


0 


0 


6 


.09 j 

i 


3 


Basic Skills 


9 


Pre 


0 


5 


k 


i 

■i 








Post 


0 


0 


9 


.02 I 




Resource Room 


6 


Pre 


0 


3 


3 


1 








Post 


0 


0 


6 


.09 1 

1 


k 


Basic Skills 


2 


Pre 


0 


0 


2 


j 








Post 


0 


0 


2 


1. 00 1 




Resource Room 


5 


Pre 


0 


2 


7 


1 




« 




Post 


0 


0 


5 


' 1 




Total Group 


38 


Pre 


0 


18 


20 










Post 


0 


0 


38 


N.C. 
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TABLE 13 

Listening for Letters, Hoijghton-Mifflin Pre-reading Inventory, Part II; 
Pre- and Post-test Distributions by Grade and Group. 



Grade 


Group 


N 


Test Time 


Score Interval 


p 










0-3 


4-11 


12-14 




2 


Basic Skills 


10 


Pre 


3 


6 


1 










Post 


0 


5 


5 


.07 




Resource Room 


6 


Pre 


1 


5 


0 










Post 


0 


1 


5 


.008 


3 


Basic Skills 


9 


Pre 


0 


9 


0 










Post 


0 


3 


6 


.004 




Resource Room 


6 


Pre 


1 


k 


1 










Post 


1 


1 


4 


.12 


k 


Basic Skills 


2 


Pre 


0 


2 


0 










Post 


0 


0 


2 


.17 




Resource Room 


5 


Pre 


0 


k 


1 










Post 


0 


2 


3 


.26 




« ' 

Total Group 


38 


Pre 


5 


30 


3 










Post 


1 


12 


20 


N.C. 





i 

1 

I 






A 






A 



TABLE Ik 



Matching Letters and Sounds, Houghton-Mifflin Pre-reading Inventory, Part II: 



?re- and Post-test Distributions by Grade and Group. 



Grade 


Group 


N 


Test Time 


Score Interval 
0-4 5-14 15-18 


P 


2 


Basic Skills 


10 


Pre 


1 


8 


1 










Post 


0 


3 


7 


.01 




Resource Room 


6 


Pre 


0 


6 


0 










Post 


0 


1 


5 


.008 



3 


Basic Skills 


9 


Pre 


0 


9 


0 










Post 


0 


4 


5 


.015 




Resource Room 


6 


Pre 


2 




1 










Post 


0 


1 


5 


.04 


4 


Basic Skills 


2 


Pre 


0 


1 


1 










Post 


0 


0 


2 


.50 




Resource Room 


5 


Pre 


0 


4 


1 










Post 


0 


1 


4 


.10 




Total Group 


38 


Pre 


3 


31 


4 










Post 


0 


10 


26 


N.C. 



O 

ERIC 
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TABLE 15 



Totol Performance Score, Houghton-Mifflin Pre-reading Inventory, Part II; 
Pre- and Post-test Distributions by Grade and Group. 



Grade 


Group 


N 


Test Time 


Score Interval 


P 










0-15 


l6-i+3 44-48 




2 


Basic .Skills 


10 


Pre 


0 


9 


1 










Post 


0 


1 


9 


.0005 




Resource Room 


7 


Pre 


0 


6 


0 










Post 


0 


1 


5 


.OOB 




Basic Skills 


9 


Pre 


0 


9 


0 










Post 


0 


0 


9 


.0002 




Resource Room 


6 


Pre 


0 


5 


1 










Post 


0 


1 


5 


.04 


k 


Basic Skills 


2 


Pre 


0 


0 


2 










Post 


0 


0 


2 


1.00 




Resource 


5 


Pre 


0 


4 


1 






« 




Post 


0 


0 


5 


.02 




Total Group 


38 


Pre 


0 


33 


5 










Ppst 


0 


s 


3^ - 


N.C. 



The probability indices shovild be interpreted, with caution, due to the small 
nu-nber of subjects in each group and the fact that Fisher's Test is mainly ap- 
plicable to examining the difference between independent groups. Despite these 
restrictions, the tobies clearly indicate the skewness of the distributions 
obtained at pre-testing. Post-test distributions do, however, indicate that 
both the resource room and basic skills laboratory activities did equally well 
in obtaining high or maximm achievement on all of the subtests. The results 
are due to at least two factors. First, the tests tap initial pre-reading and 
number skills end therefore, were the ones for which specific instructional 
objectives and procedures had been developed. The fact that the Basic Skills 
population did equally well indicates that the project has developed a package 
of objectives and coordinated instructional procedures that enable a teaching 
aide to guide children through the initial phases of learning to read. The 
second factor is related to the fact that no data is available to ascertain 
whether or not there are differential time factors or extension of skills beyond 
those sampled by the two tests. 

The extension of the objectives was apparent to the resource room teacher 
and as the pupils progressed, objectives were identified and methods emphasiz- 
ing available or newly created media were developed. Explication of the extended 
objectives and related media-instructional procedures had not been completed at 
the conclusion of the school year. The project staff anticipated extending the 
objectives and coordinated, media during the 19T1 sunnner workshop for the teachers 
and aids who would be d rectly involved in the 1971-1972 program. 

In summary, the examination of traditional methods for diagnosing children 
as m was given very little, if any, attention; a beginning was made in 
establishing a system and method for diagnosing poor performance in academic 



areas; and some progress was made in utilizing diagnostic results and identify- 
ing appropriate media to arrive at a comprehensive prescription for each child. 

The organization of documenting assessment, prescription, and instruction was 
negligible. Only by reconstructing the data provided in the second quarterly 
report (li) and coordinating the data with the objective-instruction sheets, 
which were to be completed by the end of June, could any tentative assessment 
be made of whether or not objective three had been inqplemented and been found 
effective. One confounding factor, that of requiring 100 per cent accuracy for 
each objective, was present. Many children were approximating such a level 
on the pre-test, and, thus pre-test level, instruction, and posttest scores 
could not be adequately controlled for definitive ansQysis. 

There appeared to be a limited amount of time and resources to provide for 
extensive or, at a minimum, adequate data collection. During the year key per- 
sonnel either left the project or had their responsibilities altered. In addition, 
the project's several objectives demanded attention. However, it does appear 
that a minimum amount of organlzatira is required to coordinate pre-test scores 
instruction, and post-test scores. The project's 1971 spring testing procedure 
included tests of more con 5 >lex reading skills, thereby establishing a better 
data base for assessing progress during the 1971-72 year. The progress, as it 
relates to matching objectives and instruction, should be examined in terms of 
beginning level (pre-test), acquisition time of objectives, and post-test level. 
This analysis wo\ild be possible if teachers and aides could be provided with 
seme means of efficiently checking objective selection and time expenditure. 

In addition, progress using resource rooia^ basic skills laboratories, and self- 
contained EMR instruction requires con 5 »arison with each other and with a selected 
control group. No provision has been made for this type of monitoring of progress; 



however, it does seem that such data is necessary if continued dissemination 
Is planned by the system. The efforts thus far e:^nded in development certain- 
ly warrant the empirical test^ which should also lead to empirical support. 



Ob;iective U 

Media design and modification . At the time of assessment for this report, 
the MUST project had acquired equipment for instructional and media prepara- 
tion purposes. Considerable time was spent reviewing instructional materials 
olreadj' available, a shift made by the staff once the project was underway 
nnd the emphasis was placed on the acquisition of reading skills. Instructional 
materials were identified, purchased, and modified, when needed, to fit individ- 
ualized instruction requirements. In addition, media specialists produced new 
instructional materials and a media preparation room was being established at 
C. W. Hill School (see 5^ Appendix, for detailed listing of equipment acquisitions). 
V/hile some emphasis was placed on developing and identifying a variety of types 
of media to promote attainment of instructional objectives, the coordiation of 
media, or instructional materials, with extended objectives beyond those docu- 
mented in the second quarterly report {h) was not carried out in conqolete detail. 
The plans for the project included con 5 )ilation of the developed objectives and 
instructional procedures and extension of the objectives and coordinated instruc- 
tion. This task was to be a part of the 1971 summer workshop. The data pre- 
sented in the previous section suggest Individualized instruction for the basic 
reading skills has been identified and proven effective. The objective- 
instruction sheet (See U, p.20) appears to have proven its worth, particularly 
in the Basic Skills Laboratory. Updating, extension, and evaluation of specific 
instructional materials have been slow in development, the reason being limited 
time, the pressure to plan for the 1971-72 year and the need to attend to other 
responsibilities. 
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Ob.lectlve 5 

Extension of skills resource and classroom teachers, and teaching aides 



comprising the MUST team In the three schools In the utilization of media 
identified or developed . During the 1970-71 school year only one resource 
room teacher was involved in the project and was an individual who assisted 
in the in^lementation and development of instructional objectives and pro- 
cedures. Her skills were extended to the extent that provision of a well- 
equipped media-resource room made it possible to expand her repetoire of 
instructional procedures and to work with more students than would have been 
possible with a self-contained EMR classroom. Aides have been trained to 
assist in the resource room and to manage the Basic Shills Laboratory. The 
contact with classroom teachers was informal, and usually at the initiative 
of the classroOTi teachers. Thus, development of skills of classroom teachers 
varied and no assessment was possible. At this point in time, it appears 
that extension of classroom teacher skills will depend on the successful 
inplementation of the media centers with resource room and basic skills, 
planned time for contacts between MUST team members, an^ perhaps, MUST pack- 
aging of the objectives and materials thus far developed. It should be noted 
that the most recent proposal for 1971-1972 includes the assun5)tion "Regular 
classroom teachers working cooperatively with the Resource Room Teachers and 
the MUST staff, can develop media based instructional programs which will 
permit EMR and functionally retarded pupils to spend a major portion of the 
school day in the regular classroom" (5, p.2). Actual planning for developing 
cooperation and skill extension has been lfl 5 >lied in statements concerning 
inservice programs (5). 

Objecti ves 6, 7 and 8 . No statement concerning "Significant improve- 
ment of performance in those academic areas for which individual prescriptions 
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have been written" may be made since only the basic reading skills were tested 
and for which individual prescriptions were possible (or documented). The 
students did gain, but no test v;as made to determine whether or not the pre- 
scriptive procedure was more effective than other procedures, nor was ability 
level and instruction compered with end-of-year achievement. Cost analysis 
(Objective 7) was ignored as was development of a model for national dissem- 
ination (Objective 8). However, dissemination did consume a considerable 
part of the staff's time. A review of the listing of activities (5, p.4-6) 
indicates that, in addition to the inservice work required by the project, 
workshops and/or presentations were prepared for other groups. Dissemination 
activities included approximately eight to twelve such presentations and time 
spent with visitors to the project. Time was spent preparing a film that 
covered the work undertaken by the project. The staff developed the film for 
the spring on-site visit by the funding evaluation team. The MUST staff 
expressed the belief that the film will greatly reduce the preparation time 
that had been expended during the past year for dissemination. The goals of 
local and national dissemination were, and, at the present time, are ones that 
should hold low priority. Low priority should be maintained until the tasks 
of assessment, identified instructional objectives, and media development 
have been coordinated, tested, and packaged. 

Final Summary 

The preceding discussion of the MUST project reviewed the project's 
activities and accomplishments in terms of the January, 1971, objectives. 

The goals, tasks, and products of the project altered as implementation 
progressed. In the most recent proposal, the 1970-72 activities were 



described as "decentralized management into five areas: (l) implementation 

of program at C. W. Hill School; (2) planning for future MUST involvement 
at J. M. Jones and L. P. Miles School; ( 3 ) development of a Basic Skills 
Laboratory Plan; ( 4 ) evaluation and research design; and (5) production of 
media" ( 5 , p. 2). Each of these activities has been reviewed from a slightly 
different framework. 

The development of a media center that may be utilized by all teachers 
within a school or system area is, it appears, a ma;}or luiderlying goal of 
the project. Demonstrating effective use in the area of beginning reading 
skills is the "proof" that such centers are in 5 )ortant for all areas of in- 
struction, provided one has the knowledge and skills to match multi-media 
based instruction with specific objectives. The dependence between develop- 
ment, demonstration, aad dissemination are somewhat sequential, and it appears 
that the first two require higher priority than the third. The products of 
the project, i.e. a set or series of instructional objectives coordinated 
with a variety of media-based instructional sequences, need to be "packaged" 
and tested via demonstration. The 1971-72 plans cover this aspect; however 
the documentation of the inplementation process demands analysis for dissemina- 
tion purposes. Documentation not only refers to improved data collection, but 
also to the systematic reporting of the advancement of objectives, development 
of (or identified) materials, and degree of utilization of the purchased 
produced materials and equipment. The degree to which the project staff view 
the importance of documentation remains ambiguous. 

Finally, the "systems approach" has only been partially developed in 
operation . Again, the elements and some of the linkages have been developed 
and are operating. However, the expenditure of. staff time on the components 
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of the system have tended to be imbalanced when compared with the priorities 
assigned to the different goals. This imbalance may reflect a condition of 
development and not be a factor in the 1971-72 activities. It is clear that 
the "systems approach" applies to personnel as well as program planning- 
implementation. Clear responsibilities as well as coordinated development 
of activities and policy-making are required to achieve a working system. 

In terms of the complex developmental tasks MUST has had to undertake, 
the progress made is considerable. The documentation has been less than 
desirable for optimum formative or sumative evaluation: a condition that 

has been and can be corrected dxaring the third year’s in^lementation. In 
fact, it is the 1971-72 activities that are the most appropriate for objective 
evaluation. 
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